Handling new target classes in semantic segmentation with domain adaptation

In this work, we define and address a novel domain adaptation (DA) problem in semantic scene segmentation, where the target not only exhibits data distribution shift w.r.t. source domain, but also includes classes that do exist latter. Different to “open-set” (Panareda Busto Gall, 2017) “universal adaptation” (You et al. 2019), which both regard all objects from new as “unknown”, aim at explicit test-time prediction for these classes. To reach goal, propose framework leverages zero-shot learning techniques enable “boundless” domain. It relies on architecture, along with dedicated scheme, bridge source–target gap while how map classes’ labels relevant visual representations. The performance is further improved using self-training target-domain pseudo-labels. For validation, consider different set-ups, namely synthetic-2-real, country-2-country dataset-2-dataset. Our outperforms baselines by significant margins, setting competitive standards benchmarks task. Code models are available at: https://github.com/valeoai/buda.